Process for percutaneous operations

ABSTRACT

A method is described for performing a percutaneous operation on a patient to remove an object from a cavity within the patient. The method includes advancing a first alignment sensor into the cavity through a patient lumen. The first alignment sensor provides its position and orientation in free space in real time. The alignment sensor is manipulated until it is located in proximity to the object. A percutaneous opening is made in the patient with a surgical tool, where the surgical tool includes a second alignment sensor that provides the position and orientation of the surgical tool in free space in real time. The surgical tool is directed towards the object using data provided by both the first and the second alignment sensors.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication No. 62/249,050, filed Oct. 30, 2015, which is incorporatedherein by reference.

BACKGROUND

1. Field of Art

This description generally relates to surgical robotics, andparticularly to lithotomy devices and procedures using a surgicalrobotics system.

2. Description of the Related Art

Every year, doctors perform thousands of procedures to remove urinarystones from patients' urinary tracts. Urinary stones may include kidneystones found in the kidneys and ureters as well as bladder stones foundin the bladder. Such urinary stones form as a result of concentratedminerals and cause significant abdominal pain once they reach a sizesufficient to impede urine flow through the ureter or urethra. Suchstones may formed from calcium, magnesium, ammonia, ur acid, cysteine,or other compounds.

To remove urinary stones from the bladder and ureter, surgeons use aureteroscope inserted into the urinary tract through the urethra.Typically, a ureteroscope includes an endoscope at its distal end toenable visualization of the urinary tract. The ureteroscope alsoincludes a lithotomy mechanism to capture or break apart urinary stones.During the ureteroscopy procedure, one physician controls the positionof the ureteroscope and the other surgeon controls the lithotomymechanism. The controls of the ureteroscope are located on a proximalhandle of the ureteroscope and accordingly are difficult to grasp as theorientation of the ureteroscope changes. Accordingly, presentureteroscopy techniques are labor intensive and reliant on ureteroscopeswith non-ergonomic designs.

To remove large kidney stones from the kidneys, surgeons use apercutaneous nephrolithotomy technique that includes inserting anephroscope through the skin to break up and remove the kidney stone.However, present techniques for percutaneous nephrolithotomy (“PCNL”)include using fluoroscopy to locate the kidney stone and to ensureaccurate insertion of the nephroscope. Fluoroscopy increases the cost ofthe nephrolithotomy procedure due to the cost of the fluoroscope itselfas well as the cost of a technician to operate the fluoroscope.Fluoroscopy also exposes the patient to radiation for a prolonged periodof time. Even with fluoroscopy, accurately making a percutaneousincision to access the kidney stone is difficult and imprecise.Additionally, present nephrolithotomy techniques typically involve atwo-day or three-day inpatient stay. In sum, present nephrolithotomytechniques are costly and problematic for patients.

SUMMARY

This description includes methods and devices for more easily carryingout a ureteroscopy. This description also includes methods and devicesfor more easily carrying out a PCNL. For ureteroscopy, a basketingdevice includes a number of independently manipulable pull wires thatallow full 360 degree motion of the basket, which makes capture of astone easier. A central working channel in the basketing apparatusallows a variety of other tools to be placed near a basket to break up acaptured stone. Further, a technique for ureteroscopy is described thathelps prevent stones from escaping from a basket while the basket isbeing closed.

For PCNL, a variety of techniques and devices are described that makeuse of an alignment sensor in place of fluoroscopy to detect theposition of a stone in a kidney. The alignment sensor may, for example,be an EM sensor which works in conjunction with EM field generatorsplaced around the patient and an associated CT (or other) scan toprovide position and orientation information for EM sensor in thepatient's body. The alignment sensor is placed via a cavity, such as theureter using a ureteroscope, and together with a camera is used toidentify the location of the stone. The alignment sensor provides aguidance mechanism for directing the percutaneous cut for accessing thestone within the kidney. Further, as at this point in the PCNLprocedure, a scope is already present, a working channel of the scopecan be used to advance other tools to assist in the removal of the stonethrough a port created by the PCNL. Techniques for performing the PCNLare described, as well as for how to go about removing the stone via thePCNL port.

Although this description is largely described with respect to theexample use cases of ureteroscopy, PCNL, and the removal of urinarystones and stone fragments, these descriptions are equally applicable toother surgical operations concerned with the removal of objects from thepatient, including any object that can be safely removed via a patientcavity (e.g., the esophagus, ureter, intestine, etc.) or viapercutaneous access, such as gallbladder stone removal or lung(pulmonary/transthoracic) tumor biopsy.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A shows an example surgical robotic system, according to oneembodiment.

FIG. 1B is a perspective view of a surgical robotics system withcolumn-mounted robotic arms according to one embodiment.

FIG. 2 shows an example command console for the example surgical roboticsystem 100, according to one embodiment.

FIG. 3A illustrates multiple degrees of motion of an endoscope accordingto one embodiment.

FIG. 3B is a top view of an endoscope according to one embodiment.

FIG. 3C is an isometric view of the distal end of the leader of anendoscope according to one embodiment.

FIG. 3D is an isometric view of an instrument device manipulator of thesurgical robotic system according to one embodiment.

FIG. 3E is an exploded isometric view of the instrument devicemanipulator shown in FIG. 3D according to one embodiment.

FIG. 4A is a perspective view of a surgical robotics system withcolumn-mounted arms configured to access the lower body area of asimulated patient according to one embodiment.

FIG. 4B is a top view of the surgical robotics system withcolumn-mounted arms configured to access the lower body area of thesimulated patient according to one embodiment.

FIG. 4C is a perspective view of an imaging device and a surgicalrobotics system with column-mounted arms configured to access the lowerbody area of a patient according to one embodiment.

FIG. 4D is a top view of the imaging device and the surgical roboticssystem with column-mounted arms configured to access the lower body areaof the patient according to one embodiment.

FIG. 5A is a side view of a basket apparatus according to oneembodiment.

FIGS. 5B and 5C illustrate how the basket apparatus may be used tocapture a kidney stone according to one embodiment.

FIG. 5D shows a perspective view of the robotically steerable basketapparatus, according to one embodiment.

FIG. 5E shows a planar view of the robotically steerable basketapparatus along the plane perpendicular to the center axis of the outersupport shaft, assuming it is straight, according to one embodiment.

FIG. 5F and illustrates a close up view of the distal end of the outersupport shaft of the basket apparatus, according to one embodiment.

FIG. 6A illustrates an embodiment where the basket has have a sphericalshape.

FIG. 6B illustrates an embodiment where the basket is shaped so as toform jaws.

FIG. 6C illustrates an embodiment where the basket is formed of spiralor helically shaped pull wires.

FIG. 7A illustrates insertion of a laser or optical fiber to break up acaptured stone, according to one embodiment.

FIG. 7B illustrates insertion of a mechanical drill to break up acaptured stone, according to one embodiment.

FIGS. 7C and 7D illustrate use of a chisel to break up a captured stone,according to one embodiment.

FIG. 7E illustrate use of a high pressure fluid jet to break up acaptured stone, according to one embodiment.

FIGS. 8A-8C illustrate a significant challenge that can face operatorsduring a basketing operation, according to one embodiment.

FIGS. 8D-8F illustrate a process for overcoming the challenge inbasketing a stone, according to one embodiment.

FIGS. 9A-9F illustrate a process for positioning and controlling abasket apparatus to trap stones (and stone fragments) during arobotically assisted ureteroscopy, according to one embodiment.

FIGS. 10A-10E illustrate an example of a PCNL process that includes aureteroscope including an electromagnetic sensor to identify thelocation of a stone, according to one embodiment.

FIGS. 11A-11D show example graphs illustrating on-the-fly registrationof an EM system to a 3D model of a path through a tubular network,according to one embodiment.

Reference will now be made in detail to several embodiments, examples ofwhich are illustrated in the accompanying figures. It is noted thatwherever practicable similar or like reference numbers may be used inthe figures and may indicate similar or like functionality. The figuresdepict embodiments of the described system (or method) for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles described herein.

DETAILED DESCRIPTION I. Overview

I.A. Surgical Robotics System

FIG. 1A shows an example surgical robotic system 100, according to oneembodiment. The surgical robotic system 100 includes a base 101 coupledto one or more robotic arms, e.g., robotic arm 102. The base 101 iscommunicatively coupled to a command console, which is further describedwith reference to FIG. 2. The base 101 can be positioned such that therobotic arm 102 has access to perform a surgical procedure on a patient,while a user such as a physician may control the surgical robotic system100 from the comfort of the command console. In some embodiments, thebase 101 may be coupled to a surgical operating table or bed forsupporting the patient. Though not shown in FIG. 1 for purposes ofclarity, the base 101 may include subsystems such as controlelectronics, pneumatics, power sources, optical sources, and the like.The robotic arm 102 includes multiple arm segments 110 coupled at joints111, which provides the robotic arm 102 multiple degrees of freedom,e.g., seven degrees of freedom corresponding to seven arm segments. Thebase 101 may contain a source of power 112, pneumatic pressure 113, andcontrol and sensor electronics 114—including components such as acentral processing unit, data bus, control circuitry, and memory—andrelated actuators such as motors to move the robotic arm 102. Theelectronics 114 in the base 101 may also process and transmit controlsignals communicated from the command console.

In some embodiments, the base 101 includes wheels 115 to transport thesurgical robotic system 100. Mobility of the surgical robotic system 100helps accommodate space constraints in a surgical operating room as wellas facilitate appropriate positioning and movement of surgicalequipment. Further, the mobility allows the robotic arms 102 to beconfigured such that the robotic arms 102 do not interfere with thepatient, physician, anesthesiologist, or any other equipment. Duringprocedures, a user may control the robotic arms 102 using controldevices such as the command console.

In some embodiments, the robotic arm 102 includes set up joints that usea combination of brakes and counter-balances to maintain a position ofthe robotic arm 102. The counter-balances may include gas springs orcoil springs. The brakes, e.g., fail safe brakes, may be includemechanical and/or electrical components. Further, the robotic arms 102may be gravity-assisted passive support type robotic arms.

Each robotic arm 102 may be coupled to an instrument device manipulator(IDM) 117 using a mechanism changer interface (MCI) 116. The IDM 117 canbe removed and replaced with a different type of IDM, for example, afirst type of IDM manipulates an endoscope, while a second type of IDMmanipulates a laparoscope. The MCI 116 includes connectors to transferpneumatic pressure, electrical power, electrical signals, and opticalsignals from the robotic arm 102 to the IDM 117. The MCI 116 can be aset screw or base plate connector. The IDM 117 manipulates surgicalinstruments (also referred to as surgical tools) such as the endoscope118 using techniques including direct drive, harmonic drive, geareddrives, belts and pulleys, magnetic drives, and the like. The MCI 116 isinterchangeable based on the type of IDM 117 and can be customized for acertain type of surgical procedure. The robotic 102 arm can include ajoint level torque sensing and a wrist at a distal end, such as the KUKAAG® LBR5 robotic arm. [0046]

An endoscope 118 is a tubular and flexible surgical instrument that isinserted into the anatomy of a patient to capture images of the anatomy(e.g., body tissue). In particular, the endoscope 118 includes one ormore imaging devices (e.g., cameras or other types of optical sensors)that capture the images. The imaging devices may include one or moreoptical components such as an optical fiber, fiber array, or lens. Theoptical components move along with the tip of the endoscope 118 suchthat movement of the tip of the endoscope 118 results in changes to theimages captured by the imaging devices. An example endoscope 118 isfurther described with reference to FIGS. 3A-4B in Section IV.Endoscope.

Robotic arms 102 of the surgical robotic system 100 manipulate theendoscope 118 using elongate movement members. The elongate movementmembers may include pull wires, also referred to as pull or push wires,cables, fibers, or flexible shafts. For example, the robotic arms 102actuate multiple pull wires coupled to the endoscope 118 to deflect thetip of the endoscope 118. The pull wires may include both metallic andnon-metallic materials such as stainless steel, Kevlar, tungsten, carbonfiber, and the like. The endoscope 118 may exhibit nonlinear behavior inresponse to forces applied by the elongate movement members. Thenonlinear behavior may be based on stiffness and compressibility of theendoscope 118, as well as variability in slack or stiffness betweendifferent elongate movement members.

FIG. 1B is a perspective view of a surgical robotics system 100A withcolumn-mounted robotic arms according to one embodiment. The surgicalrobotics system 100A includes a set of robotic arms 102, a set of columnrings, table 119, column 121, and base 123.

The table 119 provides support for a patient undergoing surgery usingthe surgical robotics system 100. Generally, the table 119 is parallelto the ground, though the table 119 may change its orientation andconfiguration to facilitate a variety of surgical procedures. The tablemay be rotated around the patient's transverse axis or tilted along thepatient's longitudinal axis using one or more pivots between the table119 and the column 121. The table 119 may include swivel segments,foldable segments, or both to change the configuration of an uppersurface of the table 119 that supports the patient. The table 119 mayinclude a trapdoor to facilitate drainage of bodily fluids or otherensuing fluids during surgical procedures.

The column 121 is coupled to the table 119 on one end and coupled to thebase 123 on the other end. Generally, the column 121 is cylindricallyshaped to accommodate one or more column rings 105 coupled to the column121; however, the column 121 may have other shapes such as oval orrectangular. A column ring 105 is movably coupled to the column. Forexample, a column ring 105 translates vertically along the axis of thecolumn 121, rotates horizontally around the axis of the column 121, orboth. Column rings 105 are described in more detail with respect to FIG.2 below. The column may be rotated around the column's central axisrelative to the base 123 using a rotation mechanism.

The base 123 is parallel to the ground and provides support for thecolumn 121 and the table 119. The base 123 may include wheels, treads,or other means of positioning or transporting the surgical roboticssystem 100. The base 123 may accommodate the set of robotic arms 102,the one or more column rings 105, or both as part of an inactiveconfiguration for storage, such as inside a removable housing (notshown). The base 123 may include rails (not shown) along which roboticarms 102 may be movably coupled as an alternative or supplement tocolumn rings 105.

Generally, the set of robotics arms includes one or more robotic arms102 coupled to one or more column rings 105, such as column ring 105A. Arobotic arm 102 attached to a column 105 may be referred to as acolumn-mounted robotic arm 102. The surgical robotics system 100A usesrobotic arms 102 to perform surgical procedures on a patient lying onthe table 119.

Further details and configurations regarding table 119, column 121, base123, column ring 105, and robotic arm 102 are included in U.S. patentapplication Ser. No. 15/154,765, filed May 13, 2016, as well as in U.S.patent application Ser. No. 15/154,762, filed May 13, 2016, each ofwhich is incorporated by reference herein. For example, an alternativesurgical robotics system includes a first robotic arm 102 mounted to acolumn ring 105 and a second robotic arm mounted to a rail included inthe base 123.

FIG. 2 shows an example command console 200 for the example surgicalrobotic system 100, according to one embodiment. The command console 200includes a console base 201, display modules 202, e.g., monitors, andcontrol modules, e.g., a keyboard 203 and joystick 204. In someembodiments, one or more of the command console 200 functionality may beintegrated into a base 101 of the surgical robotic system 100 or anothersystem communicatively coupled to the surgical robotic system 100. Auser 205, e.g., a physician, remotely controls the surgical roboticsystem 100 from an ergonomic position using the command console 200.

The console base 201 may include the basic components of a computersystem, that is a central processing unit (i.e., a computer processor),a memory/data storage unit, a data bus, and associated datacommunication ports that are responsible for interpreting and processingsignals such as imagery and alignment sensor data, e.g., from theendoscope 118 shown in FIG. 1. In some embodiments, both the consolebase 201 and the base 101 perform signal processing for load-balancing.The console base 201 may also process commands and instructions providedby the user 205 through the control modules 203 and 204. In addition tothe keyboard 203 and joystick 204 shown in FIG. 2, the control modulesmay include other devices, for example, computer mice, trackpads,trackballs, control pads, video game controllers, and sensors (e.g.,motion sensors or cameras) that capture hand gestures and fingergestures.

The user 205 can control a surgical instrument such as the endoscope 118using the command console 200 in a velocity mode or position controlmode. In velocity mode, the user 205 directly controls pitch and yawmotion of a distal end of the endoscope 118 based on direct manualcontrol using the control modules. For example, movement on the joystick204 may be mapped to yaw and pitch movement in the distal end of theendoscope 118. The joystick 204 can provide haptic feedback to the user205. For example, the joystick 204 vibrates to indicate that theendoscope 118 cannot further translate or rotate in a certain direction.The command console 200 can also provide visual feedback (e.g., pop-upmessages) and/or audio feedback (e.g., beeping) to indicate that theendoscope 118 has reached maximum translation or rotation.

In position control mode, the command console 200 uses athree-dimensional (3D) map of a patient and pre-determined computermodels of the patient to control a surgical instrument, e.g., theendoscope 118. The command console 200 provides control signals torobotic arms 102 of the surgical robotic system 100 to manipulate theendoscope 118 to a target location. Due to the reliance on the 3D map,position control mode requires accurate mapping of the anatomy of thepatient.

In some embodiments, users 205 can manually manipulate robotic arms 102of the surgical robotic system 100 without using the command console200. During setup in a surgical operating room, the users 205 may movethe robotic arms 102, endoscopes 118, and other surgical equipment toaccess a patient. The surgical robotic system 100 may rely on forcefeedback and inertia control from the users 205 to determine appropriateconfiguration of the robotic arms 102 and equipment.

The display modules 202 may include electronic monitors, virtual realityviewing devices, e.g., goggles or glasses, and/or other means of displaydevices. In some embodiments, the display modules 202 are integratedwith the control modules, for example, as a tablet device with atouchscreen. Further, the user 205 can both view data and input commandsto the surgical robotic system 100 using the integrated display modules202 and control modules. The display modules 202 allow for display ofgraphical GUIs that may display information about the position andorientation of various instruments operating within the patient based oninformation provided by one or more alignment sensors. This informationmay be received by electrical wires or transmitters coupled to thesensors, which transmit the information to the console base 201, whichprocesses the information for presentation via the display modules 202.

The display modules 202 can display 3D images using a stereoscopicdevice, e.g., a visor or goggle. The 3D images provide an “endo view”(i.e., endoscopic view), which is a computer 3D model illustrating theanatomy of a patient. The endo view provides a virtual environment ofthe patient's interior and an expected location of an endoscope 118inside the patient. A user 205 compares the endo view model to actualimages captured by a camera to help mentally orient and confirm that theendoscope 118 is in the correct—or approximately correct—location withinthe patient. The endo view provides information about anatomicalstructures, e.g., the shape of an intestine or colon of the patient,around the distal end of the endoscope 118. The display modules 202 cansimultaneously display the 3D model and computerized tomography (CT)scans of the anatomy the around distal end of the endoscope 118.Further, the display modules 202 may overlay the already determinednavigation paths of the endoscope 118 on the 3D model and CT scans.

In some embodiments, a model of the endoscope 118 is displayed with the3D models to help indicate a status of a surgical procedure. Forexample, the CT scans identify a lesion in the anatomy where a biopsymay be necessary. During operation, the display modules 202 may show areference image captured by the endoscope 118 corresponding to thecurrent location of the endoscope 118. The display modules 202 mayautomatically display different views of the model of the endoscope 118depending on user settings and a particular surgical procedure. Forexample, the display modules 202 show an overhead fluoroscopic view ofthe endoscope 118 during a navigation step as the endoscope 118approaches an operative region of a patient.

I.B. Endoscope

FIG. 3A illustrates multiple degrees of motion of an endoscope 118according to one embodiment. As shown in FIG. 3A, the tip 301 of theendoscope 118 is oriented with zero deflection relative to alongitudinal axis 306 (also referred to as a roll axis 306). To captureimages at different orientations of the tip 301, a surgical roboticsystem 100 deflects the tip 301 on a positive yaw axis 302, negative yawaxis 303, positive pitch axis 304, negative pitch axis 305, or roll axis306. The tip 301 or body 310 of the endoscope 118 may be elongated ortranslated in the longitudinal axis 306, x-axis 308, or y-axis 309.

The endoscope 118 includes a reference structure 307 to calibrate theposition of the endoscope 118. For example, the surgical robotic system100 measures deflection of the endoscope 118 relative to the referencestructure 307. The reference structure 307 is located on a proximal endof the endoscope 118 and may include a key, slot, or flange. Thereference structure 307 is coupled to a first drive mechanism forinitial calibration and coupled to a second drive mechanism, e.g., theIDM 117, to perform a surgical procedure.

FIG. 3B is a top view of an endoscope 118 according to one embodiment.The endoscope 118 includes a leader 315 tubular component (orleaderscope) nested or partially nested inside andlongitudinally-aligned with a sheath 311 tubular component. The sheath311 includes a proximal sheath section 312 and distal sheath section313. The leader 315 has a smaller outer diameter than the sheath 311 andincludes a proximal leader section 316 and distal leader section 317.The sheath base 314 and the leader base 318 actuate the distal sheathsection 313 and the distal leader section 317, respectively, forexample, based on control signals from a user of a surgical roboticsystem 100. The sheath base 314 and the leader base 318 are, e.g., partof the IDM 117 shown in FIG. 1.

Both the sheath base 314 and the leader base 318 include drivemechanisms (e.g., the independent drive mechanism further described withreference to FIG. 3D in Section I.C. Instrument Device Manipulator) tocontrol pull wires coupled to the sheath 311 and leader 315. Forexample, the sheath base 314 generates tensile loads on pull wirescoupled to the sheath 311 to deflect the distal sheath section 313.Similarly, the leader base 318 generates tensile loads on pull wirescoupled to the leader 315 to deflect the distal leader section 317. Boththe sheath base 314 and leader base 318 may also include couplings forthe routing of pneumatic pressure, electrical power, electrical signals,or optical signals from IDMs to the sheath 311 and leader 314,respectively. A pull wire may include a steel coil pipe along the lengthof the pull wire within the sheath 311 or the leader 315, whichtransfers axial compression back to the origin of the load, e.g., thesheath base 314 or the leader base 318, respectively.

The endoscope 118 can navigate the anatomy of a patient with ease due tothe multiple degrees of freedom provided by pull wires coupled to thesheath 311 and the leader 315. For example, four or more pull wires maybe used in either the sheath 311 and/or the leader 315, providing eightor more degrees of freedom. In other embodiments, up to three pull wiresmay be used, providing up to six degrees of freedom. The sheath 311 andleader 315 may be rotated up to 360 degrees along a longitudinal axis306, providing more degrees of motion. The combination of rotationalangles and multiple degrees of freedom provides a user of the surgicalrobotic system 100 with a user friendly and instinctive control of theendoscope 118.

FIG. 3C is a isometric view of the distal end of leader 315 of anendoscope 118 according to one embodiment. The leader 315 includes atleast one working channel 343 and pull wires and running throughconduits along the length of the walls. For example, the pull wires andmay have a helix section that helps mitigate muscling and curvealignment of the leader 315. The leader 315 includes an imaging device349 (e.g., charge-coupled device (CCD) or complementary metal-oxidesemiconductor (CMOS) camera, imaging fiber bundle, etc.), light sources350 (e.g., light-emitting diode (LED), optic fiber, etc.), and at leastone working channel 343 for other components. For example, othercomponents include camera wires, an insufflation device, a suctiondevice, electrical wires, fiber optics, an ultrasound transducer,electromagnetic (EM) sensing components, and optical coherencetomography (OCT) sensing components. In some embodiments, the leader 315includes a cavity that runs along the long axis of the leader 315 toform a working channel 343 which accommodates insertion of other devicessuch as surgical tools.

I.C. Instrument Device Manipulator

FIG. 3D is an isometric view of an instrument device manipulator 117 ofthe surgical robotic system 100 according to one embodiment. The roboticarm 102 is coupled to the IDM 117 via an articulating interface 301. TheIDM 117 is coupled to the endoscope 118. The articulating interface 301may transfer pneumatic pressure, power signals, control signals, andfeedback signals to and from the robotic arm 102 and the IDM 117. TheIDM 117 may include a gear head, motor, rotary encoder, power circuits,and control circuits. A base 303 for receiving control signals from theIDM 117 is coupled to the proximal end of the endoscope 118. Responsiveto the control signals, the IDM 117 manipulates the endoscope 118 byactuating output shafts, which are further described below withreference to FIG. 3E.

FIG. 3E is an exploded isometric view of the instrument devicemanipulator shown in FIG. 3D according to one embodiment. In FIG. 3E,the endoscope 118 has been removed from the IDM 117 to reveal the outputshafts 305, 306, 307, and 308 which may each control independent pullwires of an endoscope 118 or basket apparatus as described furtherbelow.

II. Lower Body Surgery

FIG. 4A is a perspective view of a surgical robotics system 400A withcolumn-mounted arms configured to access the lower body area of asimulated patient 408 according to one embodiment. The surgical roboticssystem 400A includes a set of robotic arms (including five robotic armsin total) and a set of three column rings. A first robotic arm 470A anda second robotic arm 470B are coupled to a first column ring 405A. Athird robotic arm 470C and a fourth robotic arm 470D are coupled to asecond column ring 405B. A fifth robotic arm 470E is coupled to a thirdcolumn ring 405C. FIG. 4A shows a wireframe of the patient 408 lying onthe table 401 undergoing a surgical procedure, e.g., ureteroscopy,involving access to the lower body area of the patient 408. Legs of thepatient 408 are not shown in order to avoid obscuring portions of thesurgical robotics system 400A.

The surgical robotics system 400A configures the set of robotic arms toperform a surgical procedure on the lower body area of the patient 408.Specifically, the surgical robotics system 400A configures the set ofrobotic arms to manipulate a surgical instrument 410. The set of roboticarms insert the surgical instrument 410 along a virtual rail 490 intothe groin area of the patient 408. Generally, a virtual rail 490 is aco-axial trajectory along which the set of robotic arms translates asurgical instrument (e.g., a telescoping instrument). The second roboticarm 470B, the third robotic arm 470C, and the fifth robotic arm 470E arecoupled, e.g., holding, the surgical instrument 410. The first roboticarm 470A and the fourth robotic arm 470D are stowed to the sides of thesurgical robotics system because they are not necessarily required tofor the surgical procedure—or at least part of the surgicalprocedure—shown in FIG. 4A. The robotic arms are configured such thatthey manipulate the surgical instrument 410 from a distance away fromthe patient 408. This is advantageous, for example, because there isoften limited space available closer toward the patient's body or thereis a sterile boundary around the patient 408. Further, there may also bea sterile drape around surgical equipment. During a surgical procedure,only sterile objects are allowed pass the sterile boundary. Thus, thesurgical robotics system 400A may still use robotic arms that arepositioned outside of the sterile boundary and that are covered withsterilized drapes to perform a surgical procedure.

In one embodiment, the surgical robotics system 400A configures the setof robotic arms to perform an endoscopy surgical procedure on thepatient 408. The set of robotic arms hold an endoscope, e.g., thesurgical instrument 410. The set of robotic arms insert the endoscopeinto the patient's body via an opening in the groin area of the patient408. The endoscope is a flexible, slender, and tubular instrument withoptical components such as a camera and optical cable. The opticalcomponents collect data representing images of portions inside thepatient's body. A user of the surgical robotics system 400A uses thedata to assist with performing the endoscopy.

FIG. 4B is a top view of the surgical robotics system 400A withcolumn-mounted arms configured to access the lower body area of thepatient 408 according to one embodiment.

FIG. 4C is a perspective view of an imaging device 440 and a surgicalrobotics system 400B with column-mounted arms configured to access thelower body area of a patient 408 according to one embodiment. Thesurgical robotics system 400B includes a pair of stirrups 420 thatsupport the legs of the patient 408 in order to expose the groin area ofthe patient 408. Generally, the imaging device 440 captures images ofbody parts or other objects inside a patient 408. The imaging device 440may be a C-arm, also referred to as a mobile C-arm, which is often usedfor fluoroscopy type surgical procedures, or another type of imagingdevice. A C-arm includes a generator, detector, and imaging system (notshown). The generator is coupled to the bottom end of the C-arm andfaces upward toward the patient 408. The detector is coupled to the topend of the C-arm and faces downward toward the patient 408. Thegenerator emits X-ray waves toward the patient 408. The X-ray wavespenetrate the patient 408 and are received by the detector. Based on thereceived X-ray waves, the imaging system 440 generates the images ofbody parts or other objects inside the patient 408. The swivel segment210 of the table 119 is rotated laterally such that the groin area ofthe patient 408 is aligned in between the generator and detector of theC-arm imaging device 440. The C-arm is a physically large device with afootprint stationed underneath the patient during use. In particular,the generator of the C-arm is disposed underneath the operative area ofthe patient, e.g., the abdomen area. In typical surgical beds mounted toa column, the column interferes with the positioning of the C-armgenerator, e.g., because the column is also underneath the operativearea. In contrast, due to the configurability of the swivel segment 210,the surgical robotics system 400B may configure the table 119 such thatthe C-arm, the robotic arms, and a user (e.g., physician) have asufficient range of access to perform a surgical procedure on a workingarea the patient's body. In one example use case, the table 119 istranslated laterally along a longitudinal axis of the table 119 suchthat the robotic arms can access the groin or lower abdomen area of apatient on the table 119. In another example use case, by rotating theswivel segment 210 away from the column 121, the generator of the C-arm440 may be positioned underneath the groin area of the patient 408. Theswivel segment 210—with a patient lying on the swivel segment 210—may berotated at least to 15 degrees relative to a longitudinal axis of thetable 119 without tipping over the surgical robotics system. Inparticular, the surgical robotics system does not tip because the centerof mass of the surgical robotics system (e.g., the center of mass of thecombined, at least, table, bed, and base) is positioned above afootprint of the base.

The surgical robotics system 400B uses a set of column-mounted roboticarms to manipulate a surgical instrument 410. Each of the robotic armsis coupled to, e.g., holding, the surgical instrument 410. The surgicalrobotics system 400B uses the robotic arms to insert the surgicalinstrument 410 into the groin area of the patient along a virtual rail490.

FIG. 4D is a top view of the imaging device 440 and the surgicalrobotics system 400B with column-mounted arms configured to access thelower body area of the patient 408 according to one embodiment.

III. Basket Apparatus

Referring now to FIGS. 5A to 5F, a robotically steerable basketapparatus is described. FIG. 5A is a side view of the basket apparatus.FIGS. 5B and 5C illustrate how the basket apparatus may be used tocapture an object, such as a urinary stone, according to one embodiment.The robotically steerable basket apparatus 500 may be operatively andremovably coupled to any of the IDMs described herein and above, such asIDM 117 described above. The robotically steerable basket apparatus 500may be advanced through a natural or artificially created orifice in asubject or patient to capture a target object within the body of thesubject or patient. For instance, the robotically steerable basketapparatus 500 may be advanced with the robotic surgical system 100through the urethra, and optionally the bladder, ureter, and/or thekidney to capture a kidney stone (ST). As another example, therobotically steerable basket apparatus 500 may be advanced into thegallbladder to capture a gallstone. In some embodiments, the roboticallysteerable basket apparatus 500 may be advanced through another workingchannel of a catheter, ureteroscope, endoscope, or similar device (e.g.,within a 1.2 mm diameter working channel). In those embodiments, theaddition of an endoscopic instrument may provide axial support andstiffness, while also delivering additional features, such as vision,navigation, and localization capabilities, to the apparatus.

The robotically steerable basket apparatus 500 may include a handle ortool base 510 adapted to removably and operatively couple with the IDM117. The tool base 510 may include a number of capstans 520 to couple tothe output shafts or drive units of the IDM so that the IDM can actuatethe capstans 520 as well as other actuation elements coupled thereto.The basket apparatus 500 further includes a number of pull wires (alsoreferred to as tendons) 530. The pull wires 530 are coupled to thecapstans 520 at one end. The pull wires 530 run straight along the longaxis of the apparatus 500, and are prevented from sagging or twisting byan outer support shaft 540. The outer support shaft 540 may include aplurality of lumens and channels through which the pull wires 530 maytraverse along the direction of the long axis of the apparatus 500. Theouter support shaft 540 may be flexible to facilitate advancement of thebasket apparatus 500 through a tortuous tissue tract or bodily channel,such as the urethra and ureter. The apparatus 500 may also include aninternal shaft 560 for axial stiffness and support. The apparatus 500may be configured to be inserted into the working channel of aninstrument such as an endoscope 118.

The pull wires 530 may be coupled to one another at the distal-most tip552 of the basket apparatus 500. For example, the basket apparatus 500may include two different pairs of pull wires 530, with each pull wirepair forming a loop with the tips of the loops coupled to one another attip 552 and each pull wire having its two ends threaded through oppositeperipheral channels or lumens 548 of the outer support shaft 540. Thetwo tips of the looped pull wires may be coupled together in any numberof ways. For example, they may be soldered together, crimped together,braided together, bonded together with an adhesive, tied together with asuture or other thread, etc. Once connected together, each pair of pullwires forming a loop can also be referred to as a single pull wire, ifthat terminology is preferred in a particular implementation.

When the tool base 510 is coupled to an IDM, the capstans 520 mayactuate the pull wires 530 so that the pull wires 530 can be translatedproximally or distally in the axial (long axis) direction, such asrelative to the outer support shaft 540. One or more of the pull wire530 may be translated independently from one another, such as by theirrespective capstans 520.

The distal ends of the pull wires 530 may extend from the distal end 544of the outer support shaft 540 to form a distal wire basket 550. Thedistal ends of the pull wires 530 may be retracted by the capstans 520located at the proximal end 542 of the outer support shaft 540 tocollapse the basket 550 into the outer support shaft 540. Retraction ofthe basket 550 into the outer support shaft 540 can lower the profile ofthe basket apparatus 500 to facilitate the advancement of the basketapparatus 500 into a tissue tract or bodily channel. In someembodiments, the apparatus 500 may be deployed through a working channelof an endoscopic device, wherein the apparatus 500 may be retractedrelative to the endoscopic device in order to similarly lower theprofile of the basket apparatus 500. Conversely, the capstans 520 may beactuated to extend the pull wires 530 out from the outer support shaft540 so that the basket 550 may expand. For instance, once the distal end544 of the outer support shaft 540 is positioned near a stone ST, thebasket 550 may be expanded to capture the stone ST.

The basket 550 may be extended from outer support shaft 540 at differentamounts of extension to vary the size of the basket 550. For instance,as illustrated in FIGS. 5B and 5C, the basket 550 may initially beextended to an enlarged size to capture the stone ST within the basket550 and then the basket 550 may be partially collapsed (i.e., reduced insize) to secure the stone within the basket 550. As further shown inFIG. 5B, the pull wires 530 may be selectively actuated to steer or tipthe basket 550 to facilitate capture of the stone ST. The outer supportshaft 540 may be held stationary relative to the pull wires 530 whilethe pull wires 530 are differentially actuated. The basket 550 may besteered in any number of directions by the differential actuation of theindividual pull wires 530 such that it has a 360° range of motion. Forexample, one end of an individual pull wire 530 may be held stationarywhile the other end is pulled or pushed to tip the basket 550 toward oraway from the moving end, respectively. In other examples, theindividual ends of the pull wires 530 may be differentially pulled,pushed, or held stationary to vary the degree and/or direction of thetipping.

The degree of movement of the capstans 520 may be indicative of thedegree and/or direction of the tipping of the basket 550 and also of itscurrent size. Therefore, in some embodiments, the robotic system, andthe IDM in particular, can determine and/or track the currentconfiguration of the basket 550 positioned within a subject or patient'sbody based on the feedback or information from the capstans 520, thedrive unit(s), or output shaft(s) and without visualization of thebasket 550. Alternatively or in combination, the basket 550 may bevisualized to determine and/or track its current configuration. The pullwires 530 may be formed from a shape memory material or metal (e.g., aNickel-Titanium alloy such as Nitinol) so that the distal ends of thepull wires 530 may be biased to assume the basket shape whenunconstrained and/or at body temperature.

FIGS. 5D, 5E, 5F illustrate different views of an embodiment that isintended for use within the working channel of an endoscopic device.FIG. 5D illustrates a perspective view of the outer support shaft of thebasket apparatus, while FIG. 5E illustrates a close up side view of theouter support shaft. As shown in FIG. 5E, the outer support shaft 540may have a square or diamond shaped cross-section. Other shapes such asa circle, ellipse, oval, triangle, quadrilateral, rectangle, pentagon,star, hexagon, and other polygonal shapes for the cross-section of theouter support shaft 540 are also contemplated. The outer support shaft540 may include a central working channel 546 and a plurality ofperipheral channels 548. The pull wires 530 may be positioned within theperipheral channels 548. A guide wire, a further therapeutic device(such as a laser fiber for lithotripsy), or a diagnostic device (such asan imaging device or a camera) may be advanced through the centralchannel 546 to reach a target area or object, such as a captured stoneST.

The outer support shaft 540 may also comprise a plurality of roundedvertices or corners 543 where the peripheral channels 548 are locatedand through which the pull wires 530 travel. Slotted lateral edges 541on the outer surface of the outer support shaft 540 may be concave, andthus at least partially curved around the corners 543 where theperipheral channels 548 are located so as to define a plurality ofelongate lateral slots or channels of the outer support shaft 540. Theseslotted lateral edges 541 may facilitate advancement of the basketapparatus 500 by discouraging apposition of tissue to the edges of theouter support shaft. When the basket apparatus 500 is positioned througha tissue tract, bodily channel, or the working channel of an endoscopicdevice, the slotted lateral edges 541 may provide sufficient space toallow fluid irrigation and/or aspiration between the outer support shaft540 and the inner walls of the tissue tract, bodily channel, or workingchannel.

FIG. 5F shows a perspective view of a robotically steerable basketapparatus, such as the device in FIG. 5D, zoomed in to illustrate thedistal end 544 of the outer support shaft 540 with the pull wiresexiting the peripheral channels 548 according to one embodiment.

FIGS. 6A-6C show example shapes for the expanded basket, according toone embodiment. When the basket 550 is expanded (e.g., not entirelycollapsed with the outer support shaft 540, or extended past the distalend of an endoscopic device), it may have any number of shapes, such asan elliptical shape as shown in FIG. 5A. As an alternative, FIG. 6Aillustrates an embodiment where the basket 550B has have a sphericalshape. As another alternative, FIG. 6B illustrates an embodiment wherethe basket 550B is shaped to have the pull wires have a semi- or fullyrigid indentation 554 so as to form the shape of “jaws” for improvedreaching capabilities within tortuous anatomy. As another alternative,FIG. 6C illustrates an embodiment where the basket 550C is formed ofspiral or helically shaped pull wires for alternative reachingcapabilities within tortuous anatomy.

FIGS. 7A-7E illustrate various techniques for using the basket apparatusto break up a captured stone, according to one embodiment. The capturedstone may be broken apart in many ways.

FIG. 7A illustrates insertion of a laser or optical fiber to break up acaptured stone, according to one embodiment. The captured stone ST maybe broken apart with laser or optical energy, referred to as laserlithotripsy. In such a use case, a laser or optical fiber 562 isintroduced from the tool base or handle 510 and advanced through thecentral working channel 546 so that a laser tip or optical element ispositioned at the proximal end of the basket 550. The central workingchannel 546 may have an appropriate size to accommodate the laser oroptical fiber, such as 100-300 μm in diameter. The laser or opticalfiber may convey laser or light energy to be directed by the laser tipor optical element to break apart the captured stone ST. Alternativelyor in combination, a fluid may be flushed through or aspirated throughthe central working channel 546. Alternatively, a fluid may be flushedthrough or aspirated through slotted lateral edges 541 as discussed withrespect to FIG. 5E.

The captured stone ST may be broken apart mechanically in many ways aswell. For example, ultrasound may be applied such as through the centralchannel 546 (not shown). Alternatively or in combination, a mechanicaldevice may be advanced through the central channel 546 and themechanical device may be used to break apart the captured stone.

FIG. 7B illustrates insertion of a mechanical drill to break up acaptured stone, according to one embodiment. The mechanical drill bit564 is advanced through the central working channel 546 of the outersupport shaft 540. A rotating motor, located proximal to the tool base510, rotates the drill bit 564 at the basket 550 to break apart thecaptured stone ST.

FIGS. 7C and 7D illustrate use of a chisel to break up a captured stone,according to one embodiment. In the embodiment of FIG. 7C, the chisel566 is advanced through the central working channel 546 of the outersupport shaft 540. The chisel 566 is actuated with a reciprocating motor(not shown) located proximal to the tool base 510. The reciprocatingmotor may drive the chisel 566 axially in the proximal and distaldirections. In the embodiment of FIG. 7D, the chisel 566 is providedfrom a device separate from the apparatus 500, and thus is not advancedthrough the basket apparatus 500, to break apart the captured stone STfrom another direction such as the distal direction. In this case, theworking channel 546 of the basket apparatus 500 may be used to evacuatestone fragments.

FIG. 7E illustrate use of a high pressure fluid jet 580 to break up acaptured stone, according to one embodiment. The high pressure fluid jet5280 may be water, saline, or another liquid ejected from a fluidsource. The fluid jet may be abrasive and comprise particulates such assalt particles to facilitate stone destruction. An exampleimplementation may have a pressure of 400 psi and a 100 μm diameter onthe central working channel 546 from which the fluid jet exits towardsthe stone ST.

The captured stone ST may be steered by the basket 550 while thecaptured stone ST is being broken apart. Once the captured stone ST isbroken apart in any of the ways described, the broken apart stone ST maybe aspirated through the central channel 546 and/or the broken apartstone ST may be secured by the basket 550 (such as in a lower profilethan if the whole stone ST were secured) to be retracted from the targetsite (e.g., ureter, renal pelvis, gallbladder, etc.). In someembodiments, the broken apart portions of the captured stone ST may beaspirated while the basket 550 continues to capture and secure thelarger portions of the captured stone ST that have not yet been brokenapart.

IV. Process for Capturing Stones in a Basket Apparatus

IV.A. Problem

Basketing is a technique frequently used by urologists to remove urinarystones or stone fragments from the urinary tract. The current state ofthe art generally requires at least two experienced operator to controlthe ureteroscope and the basket apparatus in tandem. Procedure time andclinical outcome can be negatively impacted if one or more of theoperators lack sufficient experience.

Current procedure for removing urinary stones involves advancing aureteroscope into the ureter via the urethra and bladder. Theureteroscope is positioned approximately close to a urinary stone.During a basketing phase of the operation, a basket is advanced throughthe ureteroscope and may capture the urinary stone with its basket toextract the stone. With the ureteroscope positioned at the stone, theurologist has several potential workflow options for moving the stone.If the stone is small enough that the operator is able to capture theentire stone in the basket, then both the ureteroscope and the basketare withdrawn back to the bladder or outside the subject. If the stoneis too big to withdraw in one pass, a laser (Holmium or neodymium-dopedyttrium aluminum garnet (ND:YAG)) or a electrohydraulic lithotripsy(EHL) device can be passed through a central working channel of theureteroscope and used to fragment the stone into smaller pieces. Oneoperator can then exchange the laser fiber or EHL probe for the basketand can extract each stone fragment in turn to the bladder or outsidethe patient. If the stone is located in the proximal (relative to thecenter of the body of the subject) ureter or inside the kidney itself, asheath can be used to enable rapid extraction and introduction of theureteroscope and the basket.

Such a procedure requires two operators. The primary operator controlsthe ureteroscope and the secondary operator controls the basket or anyother inserted tools such as the laser. Both the primary operator andthe secondary operator can be looking at a visual feed from aureteroscope camera.

FIGS. 8A-8C illustrate a significant challenge that can face operatorsduring a basketing operation, according tone one embodiment. In FIG. 8A,assume that the operator has advanced through the basketing phase to thepoint where the stone ST is now located within the open (i.e., unclosedor un-retracted) basket 802, by navigation of the basket, where thebasket has been advanced out of the ureteroscope 805. Despite beinglocated around the stone ST prior to retraction, retracting the basket802 so that the stone ST is trapped in the basket 802 when the basket802 is retracted is a significant challenge. This can be difficultbecause the center 811 of the basket 802 can retract laterally along thelong axis of the basket as the operator closes the basket 802. In thisscenario, if the operator initially positions the basket 802 center 811over the stone ST (FIG. 8A) and closes or collapses the basket 802 (FIG.8B), the basket 802 frequently retracts past the stone ST (see movedcenter 811 of the basket 802) and fails to trap it (FIG. 8C).

IV.B. Manual Process

FIGS. 8D-8F illustrate a process for overcoming the challenge inbasketing a stone, according to one embodiment. In this process, twooperators work together to advance the ureteroscope 805 and/or thebasket 802 in tandem as the basket 802 is closed to ensure the stone STis trapped. FIG. 8D shows the basket 802 expanded and positioned toenclose the stone ST. FIG. 8E shows the basket 802 being collapsed whileeither the ureteroscope 805 or the basket apparatus (not explicitlyshown, enclosed by the ureteroscope 805) is advanced, thereby holdingthe center 811 position of the basket in place relative to the stone ST.The ureteroscope 805 or basket apparatus may be moved relative to thepull wires through motion of either instrument as a whole by theoperator. FIG. 8F shows the stone ST securely captured or trapped by thebasket 802. After capture, the stone may be extracted from the patientby retracting the basket and/or the ureteroscope from the patient.

While the procedure above allows a stone ST to be more reliablycaptured, such a procedure typically requires two or more well trainedoperators who must work together with a high degree of coordination. Forexample, a first operator may be tasked with collapsing the basket witha first controller while a second operator may be tasked with advancingthe basket apparatus with a second controller.

IV.C. Robotic Process

Robotic control can simplify the basketing phase operation, thusreducing the complexity of the procedure, the time to perform it.Robotic control also removes the need for more than one operator to bepresent to coordinate and accomplish the basketing phase.

FIGS. 9A-9F illustrate a process for positioning and controlling abasket apparatus to trap stones (and stone fragments) during arobotically assisted ureteroscopy intervention, according to oneembodiment. The surgical robotics system 100 includes a roboticallycontrollable ureteroscope 805 which itself includes a sheath component815, a leader component (or leaderscope) 825, and a basket 802.Throughout the process, aspiration and irrigation/fluid transfer may beperformed through the space between the leader and sheath through port835, and/or they may be performed through the working channel of theureteroscope, and through the slotted edges of an inserted basketingapparatus as described with respect to FIG. 5E above.

The system further includes at least two robotic arms 102A and 102Bwhich are configured to control the position, orientation and tiparticulation of the sheath 815 and leader 825, through instrument bases801A and 801B for the sheath 815 and leader 825 respectively. In thisexample, at least one additional robotic arm 102C having a tool base102C is configured to control the position and basket actuation of thebasket 802. The system 100 may further include a graphical userinterface suitable for controlling the ureteroscope 805 and basket 802,and a control system suitable for taking command inputs from the userinterface and servoing the appropriate motor of the appropriate arm 102.The arms 102A-C, particularly their bases 801A-C, may be aligned in avirtual rail configuration.

To carry out the process of the operation, the system 100 steers therobotic ureteroscope 805 (for example, either automatically or usingoperator input received via controls and displayed via the GUI) intoposition such that the stone ST can be visualized using a tip mountedcamera (not shown) in the ureteroscope 805. As shown in FIG. 9B, thefirst 101A and second 101B instrument bases advance the sheath 815 andthe leader 825, respectively, of the ureteroscope 805 relative to thepatient in the directions indicated by a first arrow 830A and a secondarrow 830B, respectively. The speed and magnitude of the motions of thesheath 815 and leader 825 may vary from each other, and the two partsmay move independently from each other. The ureteroscope 805 is advancedthrough the bladder BL and the ureter UTR. As shown in FIG. 9C, the toolbase 801C advances the basket 802 out of the working channel of theureteroscope 805 in the direction indicated by a third arrow 830C. Inone specific implementation of this process, an introducer, such as arigid metal cystoscope, may be used to help insert the ureteroscope(sheath, leader, or both) into the patient's urethra.

As shown in FIG. 9D, the system commands the basket 802 to open andpositions the basket 802 such that the stone ST is located in the centerof the basket 802. This may be accomplished, as shown by the fourtharrow 830D, by advancing one or more of the pull wires of the basket 802using the tool base 801C, or using another arm/tool base (not shown)that separately controls the pull wires from the remainder of the basketapparatus. This may also be in part accomplished, as shown by the fiftharrow 830E, by advancing the leader 825 using the second instrument base801B, which in turn repositions the basket 802 advanced out of theleader 825.

As shown in FIG. 9E, the system then commands the basket 802 to close.This may be accomplished, as shown by the sixth arrow 830F, byretracting the pull wires of the basket 802, again using the tool base801C or using another arm/tool base (not shown) that separately controlsthe pull wires from the remainder of the basket apparatus. As the basket802 closes, either the leader's 825 instrument base 801B or the basketapparatuses' tool base (not shown) may also simultaneously advance theleader 825 or the basket 802, respectively, in the direction indicatedby seventh arrow 830G to maintain the stone ST in the center of thebasket 802 while the basket is being closed until the stone ST istrapped.

As shown in FIG. 9F, once the stone ST is trapped, the system removes orretracts the basket apparatus 802 and leader 825, according to theeighth 830H and ninth 8301 arrows using the tool base 801C andinstrument base 801B, respectively, from the subject so the stone ST canbe fully extracted from the patient.

V. Process for Percutaneous Nephrolithotomy

V.A. Problem

Some ureteral stones are sufficiently large that removal viaureteroscope is impractical. For example, stones can be greater than 2centimeters in diameter, and generally the working channel of aureteroscope through which a stone or fragment can be removed has adiameter of 1.2 millimeters. Although breaking stones into smallerfragments for removal via ureteroscopy does work in many instances,studies have shown that leftover stone debris is often the source of newstone formation, necessitating future similar treatments.

Percutaneous nephrolithotomy (PCNL), in contrast, is a process for stoneremoval whereby a surgeon cuts into the kidney from outside the body(rather than entering through the ureter) to provide a larger port forstone removal. As no ureteroscope is used to identify the location ofthe stone within a kidney, the location of the stone must be identifiedby other mechanisms. A common technique is to use traditional imagingtechniques, such as a X-ray computed tomography (CT) scan or fluoroscopyusing an intravenous pyelogram, to identify the location of the stone.

Having collected this information it is common for a urologist, who istrained to remove the stone, to ask a radiologist to perform thepercutaneous cut to place a guide wire leading near the location of thestone in the kidney out through the cut and outside the body. The cutmay be obtained by directing a nephrostomy needle into the patient'sbody, the nephrostomy needle comprising of a stylet and a cannula.Having directed the needle into the patient, the stylet may be removed,leaving the cannula to form an open port to the location of the kidneystone. Through the cannula, the urologist may then place the guide wire.The urologist can then use this wire to perform the remainder of thePCNL process to remove the stone. It is common for a urologist to ask aradiologist to place the guide wire instead of placing it themselvesbecause radiologists are specifically trained to generate and interpretCT scans, fluoroscopy scans, and other types of imaging that are used toidentify objects such as kidney stone. They are further skilled asconceptualizing the imagery information in three dimensional (3D) spaceto identify the location of an object such as a stone in that 3D space,and consequently are the most skilled at placing a guide wire accordingto that information.

To complete the PCNL, the urologist uses the placed guide wire to pass adeflated balloon or dilator along the wire. The urologist inflates theballoon or dilator to create a port large enough introduce a hollowsuction tube, such as a nephrostomy tube, directly into the calyx of thekidney containing the stone. At this point a nephroscope or any one of anumber of other instruments may be introduced into the suction tube toassist in removing the stone. For example, a stone breaker, laser,ultrasound, basket, grasper, drainage tube, etc. may be used to removethe stone or fragments thereof. Drainage tubes, such as nephrostomycatheters, may be deployed down the suction tube to reduce intra-renalpressure during and after the PCNL is completed.

PCNL is advantageous because it allows removal of larger stones thanureteroscopy, and it further allows for better flushing of leftoverstone sediment, which helps reduce new stone formation and thereforedecreases the frequency of similar follow up treatments being needed.However, PCNL is also a more aggressive treatment than ureteroscopy,requiring a minor surgery and a longer recovery period. Further, thecommon need for a radiologist to perform part of the procedure inconjunction with the urologist adds additional cost, complication, andoperation scheduling time delay to a procedure that would ideally needonly the urologist and their staff to perform. Further, PCNL requiresthe use of imaging techniques that are cumbersome and affecting on thepersons involved in the procedure. For example, fluoroscopy requires theuse of lead vests to reduce radiation uptake by hospital staff. Leadvests, however, do not eliminate all radiation, and are cumbersome towear for long periods, and over the course of an entire career can causeorthopedic injury to the staff.

V.B. Process

To address these issues, the following section describes a new processfor PCNL including an alignment sensor to identify the location of astone or the target calyx of interest. FIGS. 10A-10E illustrate anexample of this process where the alignment sensor is an electromagnetic(EM) sensor (or probe). In this process, the EM sensor is introducedinto through the bladder BL into the ureter UTR and onward into thekidney KD. The EM sensor may be attached to a ureteroscope that includesan EM sensor 1010 proximal to the tip of the ureteroscope 1005.Alternatively, the EM sensor may be as simple as a coil connected to anelectrical wire running the length of the ureteroscope which isconnected to an external computing device configured to interpretelectrical signals generated at the coil and passed down the wire.

V.B.I. Pre-Operative Segmentation & Planning

A pre-operative planning process may be performed in order to plan theprocedure and navigation of the robotic tools. The process includesperforming a pre-operative computerized tomography (CT) scan of theoperative region. The resulting CT scan generates a series oftwo-dimensional images that are used to generate a three-dimensionalmodel of the anatomical pathways and organs. The process of partitioninga CT image(s) into constituent parts may be referred to as“segmentation.” The segmented images are then analyzed by the system 100to identify the locations in three dimensional coordinate space oflandmarks within or on the surface of the patient. For PCNL, thisanalysis may include identifying landmarks including any one or more ofthe skin, kidney stone(s), bone structures (e.g., ribs, vertebrae,pelvis, etc.), internal organs (e.g., kidneys, liver, colon, etc.), andexternal devices (e.g., skin patch sensor). After segmentation iscomplete, a means of localization (such as electromagnetic detectiondiscussed below or intra-operative fluoroscopy) may be used incombination with the locations of identified landmarks and aregistration method to provide a visual representation of the locationof medical tools/instruments within the anatomy.

V.B.II. Electromagnetic Detection

Generally, an EM sensor, such as a coil, detect changes in EM fields asthe operator moves the EM sensor 1010 in the kidney KD, for example bymoving the ureteroscope tip while locating the stone ST. Animplementation of the process using an EM sensor thus further includes anumber of EM generators 1015 located externally to the patient. The EMgenerators 1015 emit EM fields that are picked up by the EM sensor 1010.The different EM generators 1015 may be modulated in a number ofdifferent ways so that when their emitted fields are captured by the EMsensor 1010 and are processed by an external computer, their signals areseparable so that the external computer can process them each as aseparate input providing separate triangulation location regarding thelocation of the EM sensor 1010, and by extension the location of thestone ST. For example, the EM generators may be modulated in time or infrequency, and may use orthogonal modulations so that each signal isfully separable from each other signal despite possibly overlapping intime. Further, the EM generators 1015 may be oriented relative to eachother in Cartesian space at non-zero, non-orthogonal angles so thatchanges in orientation of the EM sensor will result in the EM sensor1010 receiving at least some signal from at least one of the EMgenerators 1015 at any instant in time. For example, each EM generatormay be, along any axis, offset at a small angle (e.g., 7 degrees) fromeach of two other EM generators. As many EM generators as desired may beused in this configuration to assure accurate EM sensor positioninformation.

V.B.III. On-the-Fly Electromagnetic Registration

EM data is registered to an image of the patient captured with adifferent technique other than EM (or whatever mechanism is used tocapture the alignment sensor's data), such as a CT scan, in order toestablish a reference frame for the EM data. FIGS. 11A-11D show examplegraphs illustrating on-the-fly registration of an EM system to asegmented 3D model generated by a CT scan of a path through a tubularnetwork (e.g., from the bladder into a ureter into one of the kidneys),according to one embodiment.

FIGS. 11A-11D show example graphs 1110-1140 illustrating on-the-flyregistration of an EM system to a segmented 3D model of a path through atubular network, according to one embodiment. In the example of FIG.11A-11D, the EM sensor is attached to an endoscope tip 11101, howeverthe principles of registration described with respect to these figuresare equally applicable to the case where an EM sensor is attached to aguide wire and the 3D model is replaced with intra-operativefluoroscopy. In such an implementation, the 3D model discussed in thefollowing sections is replaced with fluoroscopy that updates arepresentation of the patient in with each fluoroscopy update as theguide wire is progressed through the patient. Thus, insertion of theguide wire towards the kidney and registration of the guide wire toexternal EM generators occur at least partially simultaneously.

The navigation configuration system described herein allows foron-the-fly registration of the EM coordinates to the 3D modelcoordinates without the need for independent registration prior to anendoscopic procedure. In more detail, FIG. 11A shows that the coordinatesystems of the EM tracking system and the 3D model are initially notregistered to each other, and the graph 1110 in FIG. 11A shows theregistered (or expected) location of an endoscope tip 1101 moving alonga planned navigation path 1102 through a branched tubular network (notshown here), and the registered location of the instrument tip 1101 aswell as the planned path 1102 are derived from the 3D model. The actualposition of the tip is repeatedly measured by the EM tracking system505, resulting in multiple measured location data points 1103 based onEM data. As shown in FIG. 11A, the data points 1103 derived from EMtracking are initially located far from the expected location of theendoscope tip 1101 from the 3D model, reflecting the lack ofregistration between the EM coordinates and the 3D model coordinates.There may be several reasons for this, for example, even if theendoscope tip is being moved relatively smoothly through the tubularnetwork, there may still be some visible scatter in the EM measurement,due to breathing movement of the lungs of the patient.

The points on the 3D model may also be determined and adjusted based oncorrelation between the 3D model itself, image data received fromoptical sensors (e.g., cameras) and robot data from robot commands. The3D transformation between these points and collected EM data points willdetermine the initial registration of the EM coordinate system to the 3Dmodel coordinate system.

FIG. 11B shows a graph 1120 at a later temporal stage compared with thegraph 1110, according to one embodiment. More specifically, the graph1120 shows the expected location of the endoscope tip 1101 expected fromthe 3D model has been moved farther along the preplanned navigation path1102, as illustrated by the shift from the original expected position ofthe instrument tip 1101 shown in FIG. 11A along the path to the positionshown in FIG. 11B. During the EM tracking between generation of thegraph 1110 and generation of graph 1120, additional data points 1103have been recorded by the EM tracking system but the registration hasnot yet been updated based on the newly collected EM data. As a result,the data points 1103 in FIG. 11B are clustered along a visible path1114, but that path differs in location and orientation from the plannednavigation path 1102 the endoscope tip is being directed by the operatorto travel along. Eventually, once sufficient data (e.g., EM data) isaccumulated, compared with using only the 3D model or only the EM data,a relatively more accurate estimate can be derived from the transformneeded to register the EM coordinates to those of the 3D model. Thedetermination of sufficient data may be made by threshold criteria suchas total data accumulated or number of changes of direction. Forexample, in a branched tubular network such as a bronchial tube network,it may be judged that sufficient data have been accumulated afterarriving at two branch points.

FIG. 11C shows a graph 1130 shortly after the navigation configurationsystem has accumulated a sufficient amount of data to estimate theregistration transform from EM to 3D model coordinates, according to oneembodiment. The data points 1103 in FIG. 11C have now shifted from theirprevious position as shown in FIG. 11B as a result of the registrationtransform. As shown in FIG. 11C, the data points 1103 derived from EMdata is now falling along the planned navigation path 1102 derived fromthe 3D model, and each data point among the data points 1103 is nowreflecting a measurement of the expected position of endoscope tip 1101in the coordinate system of the 3D model. In some embodiments, asfurther data are collected, the registration transform may be updated toincrease accuracy. In some cases, the data used to determine theregistration transformation may be a subset of data chosen by a movingwindow, so that the registration may change over time, which gives theability to account for changes in the relative coordinates of the EM and3D models—for example, due to movement of the patient.

FIG. 11D shows an example graph 1140 in which the expected location ofthe endoscope tip 1101 has reached the end of the planned navigationpath 1102, arriving at the target location in the tubular network,according to one embodiment. As shown in FIG. 11D, the recorded EM datapoints 1103 is now generally tracks along the planned navigation path1102, which represents the tracking of the endoscope tip throughout theprocedure. Each data point reflects a transformed location due to theupdated registration of the EM tracking system to the 3D model.

Each of the graphs shown in FIGS. 11A-11D can be shown sequentially on adisplay visible to a user as the endoscope tip is advanced in thetubular network. Additionally or alternatively, the processor can beconfigured with instructions from the navigation configuration systemsuch that the model shown on the display remains substantially fixedwhen the measured data points are registered to the display by shiftingof the measured path shown on the display in order to allow the user tomaintain a fixed frame of reference and to remain visually oriented onthe model and on the planned path shown on the display.

V.B.IV Mathematical Analysis of Registration Transform

In terms of detailed analysis (e.g., mathematical analysis) and methodsof the registration, in some embodiments, a registration matrix can beused to perform the registration between the EM tracking system and the3D model, and as one example, the matrix may represent a translation androtation in 6 dimensions. In alternative embodiments, a rotationalmatrix and a translation vector can be used for performing theregistration.

${M_{1}(\theta)} = \begin{pmatrix}1 & 0 & 0 \\0 & {\cos \; \theta} & {\sin \; \theta} \\0 & {{- \sin}\; \theta} & {\cos \; \theta}\end{pmatrix}$ ${M_{2}(\phi)} = \begin{pmatrix}{\cos \; \phi} & 0 & {{- \sin}\; \phi} \\0 & 1 & 0 \\{\sin \; \phi} & 0 & {\cos \; \phi}\end{pmatrix}$ ${M_{3}(\psi)} = \begin{pmatrix}{\cos \; \psi} & {\sin \; \psi} & 0 \\{{- \sin}\; \psi} & {\cos \; \psi} & 0 \\0 & 0 & 1\end{pmatrix}$

From a perspective view of mathematical reasoning, as one example,applying a registration transform involves a shift from one coordinatesystem (x,y,z) to a new coordinate system (x′,y′,z′) that may in generalhave its axes rotated to a different 3D orientation as well as havingits origin shifted an arbitrary amount in each dimension. For example, arotation to an azimuthal angle of radians θ may be expressed by thematrix M₁, a rotation to an inclination angle of φ radians may beexpressed by the matrix M₂ etc., and further rotational matrices may bewritten as the product of rotation matrices. Similarly, a translationvector of (Δx Δy Δz) may be chosen to represent a translation of theorigin in the x, y and z axes by Δx, Δy, and Δz respectively.

The registration transform may be determined by such methods as singularvalue decomposition on a cross correlation matrix between measured EMpositions and estimated positions in the 3D model. The transformationmatrix components may then be extracted from the decomposition, e.g., byidentifying the appropriate principle components. An error signal mayalso be generated from the residuals of the determined transform, andthe size of the error signal may be used to determine a level ofconfidence in the position. As further data are taken and theregistration transform is determined more accurately, this error signalmay decrease, indicating an increasing confidence in positions estimatedin this manner.

V.B.V. Registration Method Using Rigid Landmarks

The registration process may additionally or alternatively incorporate arigid homogenous transformation (4×4) containing a rotation matrix and atranslation vector. This transformation is obtained by a registration ofone or more point sets, typically by generating the point sets viasingle value decomposition (SVD), iterative closest point (ICP)algorithm, or another similar algorithm. For PCNL, generating point setsfor input into these algorithms may involve performing a grossregistration by (i) selecting, as a first point set, easily identifiablerigid landmarks such as ribs, ASIS of the pelvis, and vertebrae (e.g.,those identifiable on the outside of the patient) from the pre-operativeCT images during the segmentation process, and/or (ii) intraoperativelycapturing these landmarks, as a second point set, with the EMlocalization system through navigating/touching the landmarks with an EMprobe or pointer. The targeted kidney stone may also be used as alandmark. In the case of the kidney stone, the stone's location may becaptured via a EM sensor enabled ureteroscope or an EM probe attached toa guide wire. In order to reduce registration error, certain landmarksmay be weighted differently within the algorithm workflow. In caseswhere a kidney stone obstructs the renal pathways, registration usingrigid landmarks may be used independently.

In one embodiment, the registration process may includeintra-operatively capturing registration data using a combination of ahandheld EM probe, such as an embedded or “clipped-on” EM-enabled sensorto identify identifiable external rigid landmarks, such as ribs, ASIS ofpelvis, and vertebrae, that does not disturb the PCNL workflow.Calibration of the sensor or sensor-embedded device may be achieved by apivot test resulting in the correlation between the sensor's positionand the probe's tip. In some embodiments, the probe may take the formand functionality of a marker pen.

V.B.VI. Locating a Stone Based on Em and Camera Information

Referring back to FIG. 10A, once the EM sensor data has been registeredto the CT scan and the EM-enabled ureteroscope tip or guide wire with EMsensor has been advanced into the kidney KD, the operator is able tomove the ureteroscope tip (or guide wire) to identify the location of astone within the kidney or other patient organ. In the case of aureteroscope, recall from the description of the tip in Section I.Cabove with respect to FIG. 3C, the ureteroscope may include a camera forcapturing images of the field of view (FOV) in front of the tip. Cameradata captured as a video or sequence of images allows the operator tonavigate the kidney to look for the stone. Simultaneously provided EMdata from a distally coupled EM sensor in the ureteroscope identifiesthe location of the ureteroscope tip within the kidney.

In some embodiments, an EM probe or guide wire may be deployed down theworking channel of the ureteroscope to provide additional EMmeasurements for alignment. After deployment, the EM probe or guide wiremay be extended out of the working channel and past the distal tip ofthe ureteroscope in order to provide an additional EM measurement. TheEM measurement from the EM probe or guide wire may be used inconjunction with the EM data from the distally-mounted EM sensor in theureteroscope in order to generate a vector (including position andorientation information), which may be used to define a trajectory forthe percutaneous needle access to the operative region.

Once the stone has been located, for example by being present in the FOVof the camera on the tip, the percutaneous cut into the kidney can beperformed. FIG. 10B illustrates the introduction of a needle to open aport from inside the kidney KD to outside of the patient, according toone embodiment. In this process, like the ureteroscope tip or guidewire, the needle also includes an alignment sensor such as an EM sensor.Similarly to the ureteroscope or guide wire, this may be a simple coilcoupled to a wire running up the needle electrically coupled to thecomputer system. EM data received from the needle EM sensor may bereceived and processed similarly to ureteroscope tip or guide wire EMdata as describe above.

FIGS. 10C and 10D illustrate sample views of a graphical user interface1060 for visually presenting needle and ureteroscope (or guide wire) EMdata, according to one embodiment. The EM data from the needle and theEM data from the ureteroscope tip are processed together by the externalcomputing system to generate a graphical user interface that can bedisplayed to the operator to facilitate their guiding of the needletowards the stone position, as indicated by the ureteroscope EM data. Asillustrated in FIG. 10C, in one embodiment, the location of the needle,as provided by needle EM sensor data, is indicated by a first graphicalelement, such as a line on the graphical display 1060A, whereas thelocation of the stone, as indicated by ureteroscope EM data, isindicated by a second graphical element, such as a point. As illustratedin FIG. 10D, as the operator inserts the needle into the patient's bodyand moves it towards the stone, the needle graphic (line) will generallymove closer to the stone graphic (point) on the display 1060B. Thepositions of the graphics over time will indicate whether the operatoris successfully moving towards the stone, or whether they are driftingoff target. At any point, the ureteroscope may be separatelyrepositioned to re-center the stone within the FOV, move theureteroscope closer to or further from the stone, or look at the stonefrom a different angle to facilitate alignment and motion of the needletowards the stone.

The needle's motion may be constrained by the surgical robotics systemperforming the process or by design. For example, if the needle containsonly a single EM sensor, the needle may not be able to provideinformation about the roll of the needle about its long axis. In thiscase, the needle will generally be able to move in 5 degrees of freedom(in/out, pitch +/−, yaw +/−, but not roll). In one embodiment, the X, Y,and Z axes of the system (and subsequently, through the GUI, the user)of the location of the tip of the needle in space relative to the target(ureteroscope tip) and relative to the anatomy (pre-operative CT thathas been registered with the EM space and actual patient anatomy). Thepitch and yaw of the needle tip informs the system of the currentheading of the needle. With this information the system is able toproject a predicted path onto the GUI to help the physician align theneedle as he continues inserting it towards the target.

In other embodiments, additional EM sensors or other types of alignmentsensor may be added more degrees of freedom may be permitted to theneedle's motion. In yet other embodiments, the needle may be manuallydelivered by the physician using the guidance provided through therobotic system's GUI. For example, the introduction of a second EMsensor in the needle that oriented at a non-zero angle with respect tothe first EM sensor can provide a roll degree of freedom, and thesurgical robotics system 100 may be configured or designed to allow theoperator to make a roll motion.

In addition to the basic GUI introduced above, additional graphical orauditory notifications may be provided to indicate that the needle hasbeen positioned sufficiently close to the stone, that the needle hasentered the kidney, that the needle has drifted sufficiently far offcourse, or any other trigger condition that may be warranted orrequested by the operator. These notifications may change a color of thegraphical user interface, sound a tone, or otherwise. The basic GUI mayalso be more comprehensive than illustrated in FIGS. 10C and 10D. It mayalso include an outline of the kidney, which will appear differentlybased on the orientation in 3D space of the ureteroscope tip and needle.It may further include outlines of the calices of the kidney and/orvasculature surrounding the kidney, as well as outlines of other organsor critical anatomy.

FIG. 10E illustrates a point in the PCNL procedure where the needle haspenetrated the kidney near the stone, according to one embodiment. Oncethe needle has reached the stone, a balloon may be used to inflate theport, and a suction tube 1050 may be introduced to provide access to thekidney for insertion of larger diameter tools.

The PCNL process described above may be accomplished manually.Generally, the ureteroscope may be positioned first near the stone by afirst operator. The same or a different operator may then insert theneedle using the internal EM-sensor (via ureteroscope or guide wire) asa guide. In some embodiments, the EM sensor may be used to place afiducial or beacon to assist the physician to return to the location ofthe stone or calyx, so that the endoscope or guide wire can be removedand does not need to be left in the patient to accomplish the samepurpose. Alternatively, manipulation of the ureteroscope, guide wire,and needle may be accomplished by a surgical robotics system 100.

In an alternative embodiment, rather than using two different “live”alignment sensors attached to the needle and ureteroscope or guide wireas described above, the PCNL process may be carried out using only asingle “live” alignment sensor attached to the needle. In one version ofthis embodiment, the EM system is registered using a pen or otheranother implement located outside the body with an attached EM sensor.The pen is used to identify landmarks of the patient's anatomy, and isrotated to register with respect to the EM generators. With thisregistration and landmark information, an operator or the surgicalrobotics system is oriented with respect to the patient's anatomy.Subsequently, the needle can be navigated towards the kidney (or othercavity) based on data provided by the EM sensor located in the needle,the landmark location information, and the registration information. Anadvantage of this approach is that it removes the need for separatenavigation of an instrument with an EM sensor into the patient in orderto determine where to direct the needle. The loss of precision providedby the EM sensor close to the stone or other object can be at leastpartially compensated for by the landmark registration process.

The advantages of the above-described process for placing the needle andassociated port are numerous. Navigation of the needle is made lessskill intensive using the ureteroscope or guide wire as a guide. Asingle operator or robotic system may carry out the process. Fluoroscopycan be omitted if desired.

Although the above process has been described with the alignment sensorbeing an EM sensor (and associated EM generators), in practice otherkinds of positioning sensors may be used instead. Examples include, butare not limited to, accelerometers and gyroscopes, magnetometers, fiberoptic shape sensing (e.g., via Bragg gratings, Rayleigh scattering,interferometry, or related techniques), etc. Depending on theimplementation, registration to a separate form of patient imagery, suchas a CT scan, may or may not be necessary to provide a frame ofreference for locating stones within the patient.

Further, this process may be used in other operations beyond PCNL, suchas gallbladder stone removal, lung (pulmonary/transthoracic) tumorbiopsy. Generally, any type of percutaneous procedure may be performedby using an endoscope with an alignment sensor and a needle with asimilar alignment sensor. In each of these processes, the alignmentsensor-equipped endoscopic tip entered through a patient cavity into apatient organ provides a guide for the insertion of the alignmentsensor-equipped needle. Additional examples include stomach operations,esophagus and lung operations, etc. Further, the objects to be removeddo not necessarily need to be urinary stones, they may be any object,such as a foreign body or object created within the human body.

V.B.V. Stone and Fragment Removal

With the suction tube in place, a variety of techniques may be used toremove a stone. Various instruments may be inserted into the suctiontube or ureteroscope to break up or remove the stone and stonefragments. Examples include the basket apparatus described above, alaser or optical fiber to break up stones via lithotripsy, an ultrasounddevice to break up stones via shockwaves, a blender, chisel, or drill tobreak up stones mechanically, and so on. Alternatively, the variousinstruments described above may be coupled or integrated into thesuction tube in order to assist in the breaking up of kidney stonefragments and debris to assist aspiration using the suction tube.

In one embodiment, once the suction tube is in place and given that theureteroscope is already in place near the stone, any other instrumenttaking up the working channel is retracted from that ureteroscope. Thismay, for example, be the EM sensor itself or another instrument. Abasket apparatus, such as the one described above in Section III, oranother grasping tool, i.e., grasper, may then be inserted into theworking channel of the ureteroscope. and is extended out past theureteroscope tip near the stone.

The basket apparatus (or other similar device) may be used to capturethe stone and place it near the opening at the distal end of the suctiontube within the patient organ, where instruments coupled to the suctiontube, deployed down the suction tube, or deployed down the workingchannel of the ureteroscope may break up the stone in order to assistaspiration of the material down the suction tube. Additionally oralternatively, lithotripsy may be performed (using a laser tool insertedthrough the working channel of the basket apparatus, ureteroscope, ordeployed down or attached to the suction tube) to break up the stone sothat it can be sized to fit through the suction tube.

During the procedure, suction (negative pressure) may be applied downthe suction tube in order to aspirate the stone or any stone fragmentsthat are generated, while the ureteroscope or basket apparatuscontinuously irrigates the operative area. Simultaneous irrigation andsuction helps maintain pressure in the patient cavity. In embodimentswhere the ureteroscope may comprise of both a sheath component and aleader component, i.e., a leaderscope, the irrigation fluid may beprovided through the working channel of the sheath component, in orderfor the working channel of the leader component to remain available fortool deployment, such as a basket apparatus or a grasper, to helpposition and move the stone into closer proximity to the suction tubefor aspiration.

Consequently, any instruments extending out of the ureteroscope and thesuction on the suction tube operate in tandem to allow capture andremoval of the stone or stone fragments from the kidney. Due to thepresence of both the suction tube and the surgical tool, whether it be abasket apparatus or other grasping tool, the removal of the stoneeffectively proceeds as if the operator had two “hands” present withinthe kidney to deal with the removal of the stone, along withsimultaneous vision of the operating area as provided by the camera onthe tip of the ureteroscope.

VI. Additional Considerations

The processes described above, particularly those for controlling thearms of the surgical robotics system, processing alignment sensor datato generate position and orientation information for the alignmentsensor and/or needle, and for generating a graphical user interface todisplay this information may all be embodied in computer programinstructions stored within a non-transitory computer-readable storagemedium, and designed to be executed by one or more computer processorswithin one or more computing devices. The non-transitorycomputer-readable medium can be stored on any suitable computer readablemedia such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD orDVD), hard drives, floppy drives, or any suitable device. Thecomputer-executable component is preferably a processor but theinstructions may alternatively or additionally be executed by anysuitable dedicated hardware device.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs throughthe disclosed principles herein. Thus, while particular embodiments andapplications have been illustrated and described, it is to be understoodthat the disclosed embodiments are not limited to the preciseconstruction and components disclosed herein. Various modifications,changes and variations, which will be apparent to those skilled in theart, may be made in the arrangement, operation and details of the methodand apparatus disclosed herein without departing from the spirit andscope defined in the appended claims.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other. The embodiments are not limited in this context unlessotherwise explicitly stated.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the invention. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

What is claimed is:
 1. A method for performing a percutaneous operationon a patient, comprising: advancing a first alignment sensor into acavity through a patient lumen, the first alignment sensor providing theposition and orientation of the alignment sensor in free space in realtime; manipulating the first alignment sensor until the first alignmentsensor is located in proximity to an object to be removed from thecavity. making a percutaneous opening in the patient with a surgicaltool comprising a second alignment sensor that provides the position andorientation of the surgical tool in free space in real time; anddirecting the surgical tool towards the object using data provided byboth the first and the second alignment sensors.
 2. The method of claim1 wherein advancing the first alignment sensor into the cavitycomprises: advancing a distal tip of an endoscope into the cavity, thedistal tip comprising a camera to capture images of a field of view ofthe distal tip and the first alignment sensor.
 3. The method of claim 2wherein manipulating the first alignment sensor until the firstalignment sensor is located in proximity to the object comprises:manipulating the distal tip of the endoscope until the object appears inthe field of view of the camera.
 4. The method of claim 1 whereinadvancing the first alignment sensor into the cavity comprises:advancing a guide wire into the cavity, the guide wire comprising thefirst alignment sensor.
 5. The method of claim 4 wherein manipulatingthe first alignment sensor until the first alignment sensor is locatedin proximity to the object comprises: performing fluoroscopy on thepatient to generate fluoroscopic data including a location of the guidewire and first alignment sensor in the patient; and manipulating theguide wire until the first alignment sensor is in proximity with theobject based on the fluoroscopic data.
 6. The method of claim 1 whereinadvancing a guide wire into the cavity comprises: advancing aureteroscope into the cavity, the ureteroscope comprising a workingchannel; and advancing the guide wire past the distal tip of theureteroscope.
 7. The method of claim 1 wherein the first and secondalignment sensors are electromagnetic (EM) sensors that receive EMfields emitted by a plurality of EM field generators placed in proximityto the patient.
 8. The method of claim 7 wherein each of the EM sensorscomprises at least one coil of conductive material.
 9. The method ofclaim 7 further comprising: obtaining a three dimensional (3D)representation of an internal structure of the patient; and registeringdata received from the first alignment sensor to the 3D representationto determine a frame of reference for the data that aligns with aposition and orientation of the patient in free space.
 10. The method ofclaim 9 wherein the 3D representation is a CT scan.
 11. The method ofclaim 9 further comprising: segmenting the 3D representation to identifylandmarks; and registering locations of the landmarks with one or morealignment sensors to determine the frame of reference.
 12. The method ofclaim 9: wherein registering the data comprises: aggregating thelocations of the landmarks into at least one point set; and determining,based on point set, a homogeneous transformation comprising a rotationmatrix and a translation vector.
 13. The method of claim 12: wherein atleast one of the landmarks is identifiable on an outside of the patient;and wherein the first alignment sensor used to register locations of thelandmarks is navigated outside the patient to register the locations.14. The method of claim 13: wherein the first alignment sensor used toregister locations of the landmarks is coupled to a hand-held implement.15. The method of claim 12: wherein at least one of the landmarks isidentifiable intra-operatively; and wherein the first alignment sensorused to register locations of the landmarks is navigated inside thepatient to register the locations.
 16. The method of claim 7 furthercomprising: obtaining a three dimensional (3D) representation of aninternal structure of the patient; and navigating the first alignmentsensor outside the patient to identify a first point set comprisinglocations of landmarks identifiable outside the patient; navigating asecond alignment sensor inside the patient to identify a second pointset comprising locations of landmarks identifiable inside the patient;registering the first and second point sets to the 3D representation todetermine a frame of reference for the data that aligns with a positionand orientation of the patient in free space.
 17. The method of claim 1wherein each alignment sensor is electrically coupled to a conductivewire which transmits sensor data to a computing system for processing.18. The method of claim 17, further comprising: receiving, at thecomputer system, data from the first and second alignment sensors inreal time; providing, via a display device, a graphical interfacedisplaying a first graphical element representing the position andorientation of the distal tip, and a second graphical elementrepresenting the position and orientation of the surgical tool.
 19. Themethod of claim 18, wherein directing the surgical tool towards theobject using data provided by both the first and the second alignmentsensors comprises: updating the display of the first graphical elementin the graphical interface in response to motion of the distal tip ofthe endoscope within the patient.
 20. The method of claim 18, whereindirecting the surgical tool towards the object using data provided byboth the first and the second alignment sensors comprises: updating thedisplay of the second graphical element in the graphic interface inresponse to motion of the surgical tool within the patient.
 21. Themethod of claim 18, wherein directing the surgical tool towards theobject using data provided by both the first and the second alignmentsensors comprises: providing a notification that the surgical tool hasarrived at the object responsive to the first and the second alignmentsensors being in sufficiently close proximity.
 22. The method of claim 1wherein the steps of advancing the distal tip of the endoscope,manipulating the distal tip of the endoscope, making the percutaneousopening, and directing the surgical tool towards the object arecontrolled by a plurality of robotic arms.
 23. The method of claim 1wherein advancing the distal tip of the endoscope comprises: receivingan input at a control module; and actuating the plurality of roboticarms in response to the second input, the actuation of the robotic armsadvancing the distal tip of the endoscope.
 24. The method of claim 23,wherein directing the surgical tool comprises: receiving a second inputat a control module; and actuating a plurality of robotic arms inresponse to the input, the actuation of the robotic arms directing thesurgical tool.
 25. The method of claim 1 wherein the surgical tool is aneedle comprising a balloon, the method further comprising: inflatingthe balloon to create a port into the patient cavity; and inserting asuction tube into the port, the suction tube having a diametersufficient to remove the object or fragments thereof.
 26. The method ofclaim 25, the method further comprising: advancing an endoscope into thecavity through a patient lumen other than the port, the endoscopecomprising a working channel; irrigating the patient cavity with afluid, where the fluid passes within the working channel of theendoscope; and applying negative pressure to the suction tube, thecombination of the fluid irrigation and the negative pressure assistingin causing the object to be removed from the cavity through the suctiontube.
 27. The method of claim 26, the method further comprising at leastone of: passing an optical fiber or a laser through the working channeland activating the optical fiber or laser to perform lithotripsy on theobject to break it apart; and passing an endoscopic tool through theworking channel and manipulating the endoscopic tool to position theobject to be removed from the cavity through the suction tube.
 28. Themethod of claim 27, wherein the endoscopic tool is at least one of abasket apparatus and a grasper.
 29. The method of claim 1 wherein theobject is at least one from the group consisting of: a kidney stone or abladder stone, a stone formed from at least one of calcium, magnesium,ammonia, uric acid, and cysteine.
 30. The method of claim 1 wherein thecavity is a kidney of a patient or a calyx within the kidney of thepatient.